Video data generation apparatus, video data generation method, and program

ABSTRACT

A frame rate determination unit determines a frame rate of video data. A code amount determination unit determines a target code amount that can be used in each frame of the video data. An encoding unit performs generation processing for generating video data according to the frame rate and the code amount. When the frame rate determination unit changes the frame rate from a first frame rate to a second frame rate, the code amount determination unit makes at least a target code amount in a frame immediately before the first frame smaller than when the frame rate is not changed from the first frame rate to the second frame rate in the first frame, and makes a target code amount in the first frame greater than when the frame rate is not changed from the first frame rate to the second frame rate in the first frame.

BACKGROUND OF THE INVENTION Field of the Invention

The present disclosure relates to a video data generation apparatus, avideo data generation method, and a program.

Description of the Related Art

Conventionally, a network camera that distributes a moving image (video)through a network uses a method of suppressing an amount of code of thevideo to be distributed by deleting a redundant part for a user. As themethod of suppressing the amount of code of the video to be distributed,a method of controlling a frame rate of the video to be distributedaccording to the presence or absence of a moving object in the video isknown.

Japanese Patent Laid-Open No. 2006-115470 discloses a technology thatdetects a moving object from a video and determines a frame rate byevaluating smoothness of motion of the moving object.

In the technology described in Japanese Patent Laid-Open No.2006-115470, when an evaluation value for evaluating the smoothness ofmotion of the moving object is greater than a specified value, the framerate is set to a slow frame rate, and when the evaluation value issmaller than the specified value, the frame rate is set to a fast framerate. However, in the technology described in Japanese Patent Laid-OpenNo. 2006-115470, the image quality of an image distributed at the slowframe rate and the image quality of an image distributed at the fastframe rate are treated equally.

When the frame rate is the slow frame rate, screen update is slower thanwhen the frame rate is the fast frame rate, so that if the image qualityof the distributed image is low, the low quality image is presented to auser over a long time, and an impression that the image quality isdegraded is presented to the user.

SUMMARY OF THE INVENTION

Therefore, in order to suppress degradation of video image quality whenthe frame rate is controlled to a second frame rate lower than a firstframe rate while realizing reduction of the entire amount of code of thevideo by frame rate control, the present disclosure has, for example,the following configuration.

A video data generation apparatus, which performs generation processingfor generating video data, includes a frame rate determination unit thatdetermines a frame rate of the video data, a code amount determinationunit that determines a target code amount that can be used in each frameof the video data, and an encoding unit that performs the generationprocessing for generating the video data according to the frame ratedetermined by the frame rate determination unit and the code amountdetermined by the code amount determination unit. When the frame ratedetermination unit changes, in a first frame, the frame rate of thevideo data from a first frame rate to a second frame rate lower than thefirst frame rate, the code amount determination unit makes at least atarget code amount in a frame immediately before the first frame smallerthan when the frame rate is not changed from the first frame rate to thesecond frame rate in the first frame, and makes a target code amount inthe first frame greater than when the frame rate is not changed from thefirst frame rate to the second frame rate in the first frame.

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a block configuration example of an imageprocessing apparatus according to an embodiment.

FIG. 2 is a hardware configuration example of the image processingapparatus.

FIG. 3 is a conceptual diagram of moving image compression according toa first embodiment.

FIG. 4 is a flowchart showing a procedure of moving image compressionprocessing of the first embodiment.

FIG. 5 is a conceptual diagram of moving image compression according toa second embodiment.

FIG. 6 is a flowchart showing a procedure of moving image compressionprocessing of the second embodiment.

FIG. 7 is a block configuration diagram showing details of a compressioncontrol unit of a third embodiment.

FIG. 8 is a conceptual diagram of moving image compression according tothe third embodiment.

FIG. 9 is a flowchart showing a procedure of moving image compressionprocessing of the third embodiment.

FIG. 10 is a diagram for explaining a frame rate control for each area.

FIG. 11 is a block configuration diagram showing details of acompression control unit of a fourth embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments for carrying out the present disclosure will bedescribed in detail with reference to the drawings.

The embodiments described below are examples of an implementation unitof the present disclosure and should be appropriately corrected orchanged according to a configuration and various conditions of anapparatus to which the present disclosure is applied. The presentdisclosure is not limited to the embodiments described below.

FIG. 1 is a diagram showing a block configuration example of an imageprocessing apparatus (video data generation apparatus) 100 according toan embodiment.

The image processing apparatus 100 is a moving image compressionapparatus that compresses a moving image (video) composed of consecutiveframes on a time-series. In the present embodiment, a case will bedescribed where a network camera used as an abnormality monitoringsystem operates as the image processing apparatus 100. The networkcamera captures an image of a predetermined monitoring area anddistributes a captured video to a client apparatus through a network. Inthis case, the network camera detects a moving object from the capturedvideo, determines a frame rate of the video to be distributed based onmoving object information related to the detected moving object, andperforms a frame rate control that controls the frame rate.

As shown in FIG. 1, the image processing apparatus 100 includes a videoinput unit 101, a compression unit 102, a moving object detection unit103, an FR control unit 104, a compression control unit 105, and adistribution unit 106.

The video input unit 101 acquires video data to be compressed anddistributed and outputs the acquired video data to the compression unit102 and the moving object detection unit 103.

The compression unit 102 compresses the video inputted from the videoinput unit 101 according to a compression control of the compressioncontrol unit 105 and outputs compressed data to the FR control unit 104.Here, a compression method of the video may be a compression methodusing in-frame correlation (in-frame prediction), a compression methodusing interframe correlation (interframe prediction), or a compressionmethod combining these methods. For example, as the compression methodof the video, it is possible to use an encoding method compatible withH.264. However, the compression method of the video is not limited tothe above methods, but any compression method may be used.

The moving object detection unit 103 detects a moving object from thevideo inputted from the video input unit 101 and outputs moving objectinformation, which is a result of the moving object detection, to the FRcontrol unit 104. For example, the moving object detection unit 103 canuse a background subtraction where a difference between an inputtedimage (frame) and a background image created from an image inputted inthe past is detected as a moving object.

The FR control unit 104 determines the frame rate of the video to bedistributed based on moving object information inputted from the movingobject detection unit 103. The FR control unit 104 determines the framerate by selecting a frame rate mode from, for example, the followingthree modes based on the moving object information.

(1) Fast frame rate mode: A mode where a moving object exists in avideo. The frame rate may be a fixed value such as, for example, 30 fps(frames/sec) or may be variable according to velocity of motion of themoving object. Hereinafter, the frame rate in the fast frame rate modeis referred to as “fast frame rate”, and the fast frame rate mode isreferred to as “fast mode”.

(2) Slow frame rate mode: A mode where no moving object exists in avideo. The frame rate may be a value lower than the fast frame rate and,for example, may be 0.1 fps or 0.001 fps. Moreover, the frame rate maybe 0 fps. Here, 0 fps may be simulatively achieved by sending a pseudoframe including no image information. Hereinafter, the frame rate in theslow frame rate mode is referred to as “slow frame rate”, and the slowframe rate mode is referred to as “slow mode”.

(3) Transition frame rate mode: A mode where a frame rate is graduallyreduced over a predetermined time period when it is determined that animage where a moving object exists transits to an image where no movingobject exists and the frame rate should be switched from the fast framerate to the slow frame rate. Here, the predetermined time period may beset according to statistic information of moving object detection(detection frequency of a moving object) or may be uniformly set to, forexample, two seconds or the like. The frame rate may be a function thatconnects the fast frame rate and the slow frame rate with a linear orlogarithmic function and temporally varies between them or may simply bea constant between the slow frame rate and the fast frame rate.Hereinafter, the frame rate in the transition frame rate mode isreferred to as “transition frame rate” and the transition frame ratemode is referred to as “transition mode”.

The compression control unit 105 is inputted with the frame rate modedetermined by the FR control unit 104 and controls the compression unit102 according to the inputted mode. Specifically, the compressioncontrol unit 105 sets a quantization value (Q value) representing aquantization amount when compressing an image according to the framerate mode inputted from the FR control unit 104 and outputs thequantization value to the compression unit 102. At this time, thecompression control unit 105 controls the quantization value to improveimage quality in the slow mode by increasing the amount of code of videoto be distributed at the slow frame rate to be greater than a specifiedvalue within a range where the entire amount of code of the video to bedistributed does not exceed a target amount of code. Details of acompression control method of the compression control unit 105 will bedescribed later.

The distribution unit 106 distributes a compressed video to a networknot shown in the drawings according to the frame rate determined by theFR control unit 104.

FIG. 2 is a hardware configuration example of the image processingapparatus 100.

The image processing apparatus 100 includes a CPU 11, a ROM 12, a RAM13, an external memory 14, a communication I/F 15, and a system bus 16.

The CPU 11 integrally controls operation of the image processingapparatus 100. The ROM 12 is a non-volatile memory that stores a controlprogram necessary for the CPU 11 to perform processing. The program maybe stored in the external memory 14 or an attachable/detachable storagemedium not shown in the drawings. The RAM 13 functions as a main memory,a work area, and the like of the CPU 11. The CPU 11 loads necessaryprogram and the like from the ROM 12 to the RAM 13 when performingprocessing and implements various functional operations by executing theprogram and the like.

The external memory 14 stores, for example, various data, variousinformation, and the like that are required when the CPU 11 performsprocessing using a program. Further, the external memory 14 storesvarious data, various information, and the like that are obtained whenthe CPU 11 performs processing using a program. The communication I/F 15provides a wired or wireless communication interface with an externaldevice.

Functions of some or all of elements of the image processing apparatus100 shown in FIG. 1 can be realized when the CPU 11 executes a program.However, at least some of the elements of the image processing apparatus100 shown in FIG. 1 may operate as dedicated hardware. In this case, thededicated hardware operates under control of the CPU 11.

In the present embodiment, the network camera operates as the imageprocessing apparatus 100. Therefore, in the present embodiment, theimage processing apparatus 100 can include an image capturing unit as ahardware configuration. Here, the image capturing unit includes a lensunit and an image capturing element that constitute an image captureoptical system. The image capturing element includes a CCD sensor, aCMOS sensor, or the like and converts an image formed on a lightreceiving surface of the image capturing element into an electricalsignal.

In the present embodiment, a case will be described where the networkcamera operates as the image processing apparatus 100. However, a usualPC or another device may operate as the image processing apparatus 100.

Hereinafter, the compression control method of the compression controlunit 105 will be specifically described.

FIG. 3 is a conceptual diagram of moving image compression according tothe present embodiment and is a diagram where the vertical axisrepresents an amount of code of each image (frame) distributed by theimage processing apparatus 100 and the horizontal axis represents time.

When a moving object exists in a video, the image processing apparatus100 distributes the video in the fast mode. When the image processingapparatus 100 detects that no moving object exists in the video, theimage processing apparatus 100 transits from the fast mode to thetransition mode and thereafter transits to the slow mode. FIG. 3 shows acase where the image processing apparatus 100 detects that no movingobject exists in the video at time T1.

In this way, when the image processing apparatus 100 detects a movingobject from the video, the image processing apparatus 100 controls theframe rate to the fast frame rate and presents a user with a smoothmotion of a subject. At this time, the image processing apparatus 100can determine the fast frame rate according to the velocity of themoving object. When the image processing apparatus 100 detects no movingimage from the video, the image processing apparatus 100 switches theframe rate from the fast frame mode to the slow frame mode andsuppresses the amount of code by deleting redundant portions for theuser. Further, the image processing apparatus 100 sets the transitionmode and transits to the transition mode when moving from the fast modeto the slow mode. Thereby, it is possible to cope with a video in whicha change between static and dynamic is extreme such as a video in whicha state where there is a motion of a subject changes to a state wherethere is no motion of the subject and immediately thereafter returns toa state where there is a motion of the subject.

An amount of code 301 is an amount of code of a frame to be distributedin the fast mode. When a motion of the subject disappears in a framedistributed at time T1, thereafter the frame rate mode becomes thetransition mode. In the transition mode, a transitional frame rate andan amount of code for causing the image processing apparatus 100 totransit from the fast mode to the slow mode are set. As described above,the transition frame rate may be a fixed value or, for example, may bemade to gradually approach the slow frame rate from the fast frame rateby a logarithmic function.

An amount of code 302 of a frame to be distributed in the transitionmode is more suppressed than the amount of code 301 in the fast mode.The compression control unit 105 of the image processing apparatus 100strongly performs compression by making a quantization value in thetransition mode greater than a quantization value in the fast mode by,for example, three, and suppresses the amount of code. The compressioncontrol unit 105 suppresses the amount of code as described above fromthe first frame to a frame immediately before the last frame of thetransition mode.

The compression control unit 105 assigns the amount of code suppressedin the transition mode to the last frame of the transition mode.Specifically, the compression control unit 105 totals differential codeamounts 303 between an average of the amounts of code 301 in the fastmode and the amount of code 302 in the transition mode of frames fromthe first frame to the frame immediately before the last frame of thetransition mode. The compression control unit 105 controls aquantization value for the last frame so that a total value of thedifferential code amounts 303 is assigned to the last frame.

Thereby, an amount of code 304 of the last frame becomes a value greaterthan an amount of code 306 that is a specified value in the slow mode bya total code amount 305 shown by oblique lines. Therefore, it ispossible to transit to the slow mode in a state where the image qualityis improved. Here, the specified value in the slow mode is an amount ofcode of a frame compressed according to a quantization value for theslow frame rate. In the present embodiment, the quantization value forthe slow frame rate may be the same as a quantization value for the fastframe rate.

Next, an operation of the image processing apparatus 100 according tothe present embodiment will be described.

FIG. 4 is a flowchart showing a procedure of moving image compressionprocessing performed by the image processing apparatus 100. The movingimage compression processing is processing that improves image qualityin the slow mode when performing frame rate control that changes a framerate based on moving object information in a video. The processing ofFIG. 4 is started at a timing when the image processing apparatus 100 isstarted. However, a start timing of the processing of FIG. 4 is notlimited to the aforementioned timing. The image processing apparatus 100can implement each processing shown in FIG. 4 when the CPU 11 reads andexecutes a necessary program. Hereinafter, an alphabet character “S”means a step in the flowchart.

First, in S1, the FR control unit 104 performs initialization andselects the fast mode as the frame rate mode.

Next, in S2, the FR control unit 104 is inputted with the moving objectinformation from the moving object detection unit 103 and determineswhether a moving object is detected from a video. When the moving objectis detected, the FR control unit 104 proceeds to S3, and when no movingobject is detected, the FR control unit 104 proceeds to S8. In S3, theFR control unit 104 determines whether a current frame rate mode is theslow mode. When determining that the current frame rate mode is not theslow mode, the FR control unit 104 proceeds to S4, and when determiningthat the current frame rate mode is the slow mode, the FR control unit104 proceeds to S6.

In S4, the FR control unit 104 determines whether the current frame ratemode is the transition mode. When determining that the current framerate mode is not the transition mode, the FR control unit 104 directlyreturns to S2 because the current frame rate mode is the fast mode. Onthe other hand, when determining that the current frame rate mode is thetransition mode in S4, the FR control unit 104 proceeds to S5. In S5,the compression control unit 105 initializes the quantization value forthe last frame and proceeds to S6. Here, the quantization value for thelast frame is a quantization value to be applied to the last frame ofthe transition mode.

In S6, the compression control unit 105 sets the quantization value forthe fast frame rate to the compression unit 102 and proceeds to S7. InS7, the FR control unit 104 sets the frame rate to the fast frame rate,transits to the fast mode, and returns to S2.

In S8, the FR control unit 104 determines whether the current frame ratemode is the fast mode. When determining that the current frame rate modeis the fast mode, the FR control unit 104 proceeds to S9, and whendetermining that the current frame rate mode is not the fast mode, theFR control unit 104 proceeds to S11. In S9, the FR control unit 104 setsthe frame rate to the transition frame rate, transits from the fast modeto the transition mode, and proceeds to S10. In S10, the FR control unit104 starts a timer for ending the transition mode and returns to S2.

In S11, the FR control unit 104 determines whether the current framerate mode is the transition mode. When determining that the currentframe rate mode is the transition mode, the FR control unit 104 proceedsto S12, and when determining that the current frame rate mode is not thetransition mode, the FR control unit 104 proceeds to S17.

In S12, the FR control unit 104 determines whether a frame to beprocessed is the last frame of the transition mode. The FR control unit104 can determine whether the frame to be processed is the last frame ofthe transition mode based on the timer set in S10. When determining thatthe frame to be processed is not the last frame of the transition mode,the FR control unit 104 proceeds to S13, and the compression controlunit 105 sets a quantization value for the transition frame rate to thecompression unit 102 and proceeds to S14.

In S14, the compression control unit 105 updates the quantization valuefor the last frame and returns to S2. In S14, the compression controlunit 105 accumulates a difference between an amount of code of a framecompressed according to the quantization value for the transition framerate and an amount of code of a frame compressed according to thequantization value for the fast frame rate. Then, the compressioncontrol unit 105 obtains the quantization value for the last frame sothat the accumulated differential code amount becomes an amount of codeto be added to the amount of code of the last frame and updates thequantization value for the last frame. The quantization value Qf for thelast frame can be calculated based on, for example, the followingformula.

$\begin{matrix}\left\lbrack {{Expression}\mspace{14mu} 1} \right\rbrack & \; \\{{Qf} = {{Qk} + {\log_{\beta}\left( {1 + \frac{\alpha\;\sigma_{k = 0}^{n}{bk}}{a}} \right)}}} & (1)\end{matrix}$

Here, α and β are efficiency, a is an average of amounts of code perframe in the transition mode, n is the number of frames in thetransition mode, bk is a difference between an average of amounts ofcode per frame in the fast mode and an amount of code of a k-th frame inthe transition mode, and Qk is a quantization value in the transitionmode. Here, α is an efficiency when the differential code amounts of aplurality of frames are pushed into one frame and is, for example, 0.3.Here, β is a value representing an efficiency of quantization such as adegree of change in the amount of code when the quantization value isincremented by 1 and is, for example, 0.88. Regarding α and β, it ispossible to dynamically calculate prediction values for themrespectively. In this case, it is possible to dynamically calculate theprediction values based on, for example, initial values of α=0.3 andβ=0.88, an obtained amount of code, a set quantization value, and thenumber of transition frames n by using, for example, a Kalman filter.

In S12, when determining that the frame to be processed is the lastframe of the transition mode, the FR control unit 104 proceeds to S15,and the compression control unit 105 sets the quantization value for thelast frame that is finally updated in S14 to the compression unit 102and proceeds to S16. In S16, the FR control unit 104 sets the frame rateto the slow frame rate, transits from the transition mode to the slowmode, and returns to S2.

In S17, the compression control unit 105 sets the quantization value forthe slow frame rate to the compression unit 102 and proceeds to S18. InS18, the compression control unit 105 initializes the quantization valuefor the last frame and returns to S2.

As described above, the image processing apparatus 100 according to thepresent embodiment acquires a video composed of consecutive frames on atime-series and compresses and distributes the acquired video. At thistime, the image processing apparatus 100 detects a moving object fromthe video, determines a frame rate at which the compressed video isdistributed based on moving object information related to the detectedmoving object, and distributes the video according to the determinedframe rate. Further, the image processing apparatus 100 controls thecompression unit 102 and increases the amount of code of the frame to bedistributed at the slow frame rate to be greater than the specifiedvalue within a range where the amount of code of the video to bedistributed does not exceed the target amount of code.

Thereby, the image processing apparatus 100 can improve the imagequality at the slow frame rate while realizing reduction of the amountof code of the video by the frame rate control.

The image processing apparatus 100 has the fast frame rate mode (fastmode), the slow frame rate mode (slow mode), and the transition framerate mode (transition mode) as the frame rate mode. The image processingapparatus 100 increases the amount of code of the last frame among theframes to be distributed at the transitional frame rate to be greaterthan the specified value, so that the image processing apparatus 100increases the amount of code of the first frame to be distributed at theslow frame rate to be greater than the specified value.

In this way, the amount of code of the last frame of the transitionmode, which is the last frame of the frames during a period changingfrom the fast mode to the slow mode, is increased, so that it ispossible to appropriately improve the image quality of the first frameof the slow mode.

Further, the image processing apparatus 100 controls the compressionunit 102 and suppresses the amounts of code of frames except for thelast frame among the frames to be distributed at the transitional framerate, and appropriates the suppressed amounts of code for the lastframe.

In this way, the amounts of code of the frames of the transition modeare suppressed by quantization value control and the suppressed amountsof code are appropriated for the last frame. Therefore, it is possibleto improve the image quality at the slow frame rate without increasingthe entire amount of code of the video from the amount of code (thetarget amount of code) in a case where the quantization value control isnot performed.

As described above, in the present embodiment, it is possible to improvethe image quality at the slow frame rate when the frame rate control isperformed. Therefore, while the reduction of the entire amount of codeof the video is realized by the frame rate control, it is possible tohardly give an impression of degradation of image quality of the videoon a user when the control is transited to the slow frame rate lowerthan the fast frame rate.

Second Embodiment

Next, a second embodiment of the present disclosure will be described.

In the second embodiment, the frame rate of the transition mode isadjusted so that the last frame of the transition mode becomes a framecompressed by using the in-frame correlation (in-frame prediction).

FIG. 5 is a conceptual diagram of moving image compression according tothe present embodiment and is a diagram where the vertical axisrepresents an amount of code of each image (frame) distributed by theimage processing apparatus 100 and the horizontal axis represents time.

In the present embodiment, as a compression method, a compression methodusing the in-frame correlation (in-frame prediction) and the interframecorrelation (interframe prediction) is adopted. Therefore, in thepresent embodiment, as a result of compression, two types of frames: aframe (I frame) compressed by using the in-frame correlation and a frame(P frame) compressed by using the interframe correlation are outputted.In general, the amount of code of the I frame is greater than the amountof code of the P frame. Further the quality of image of the I frame isgenerally better than that of the P frame.

In FIG. 5, a code amount 311 represents the amount of code of the Iframe distributed in the fast mode, and a code amount 312 represents theamount of code of the P frame distributed in the fast mode. When amotion of a subject disappears in a frame distributed at time T2, theframe rate mode transits from the fast mode to the transition mode, andthereafter the frame rate mode transits to the slow mode.

In the transition mode, the quantization value is controlled so that adifference between a code amount 313 of the P frame of the transitionmode and an average code amount of the P frame of the fast mode becomesa code amount 314. Regarding the I frame, similarly, the quantizationvalue is controlled so that a difference between a code amount 315 ofthe I frame of the transition mode and an average code amount of the Iframe of the fast mode becomes a code amount 316.

The compression control unit 105 assigns the amount of code suppressedin the transition mode to the last frame of the transition mode.Specifically, the compression control unit 105 adds a total ofdifferential code amounts of P frames and a total of differential codeamounts of I frames of frames from the first frame to the frameimmediately before the last frame of the transition mode. Then, thecompression control unit 105 controls the quantization value of the lastframe so that a total code amount 318 obtained by the addition isappropriated to the last frame. Here, the last frame of the transitionmode is assumed to be an I frame. Thereby, a code amount 317 of the lastframe is an amount obtained by adding the total code amount 318 shown byoblique lines to a code amount 319 which is a specified value in theslow mode. Here, the specified value is an amount of code of an I framecompressed according to a quantization value for the slow frame rate.

At this time, the FR control unit 104 determines and controls the framerate of the transition mode so that the last frame of the transitionmode becomes the I frame.

Although the last frame of the transition mode is set to the highestimage quality, if there still remains the amount of code to beappropriated, the amount of code may be limited.

The configuration of the image processing apparatus according to thepresent embodiment is similar to that of the image processing apparatus100 shown in FIG. 1. However, processing of the FR control unit 104 isdifferent from that of the first embodiment. Therefore, processingdifferent from that of the first embodiment will be mainly describedbelow.

The FR control unit 104 determines the frame rate of the video to bedistributed based on the moving object information inputted from themoving object detection unit 103. In the same manner as in the firstembodiment, the FR control unit 104 selects the frame rate from amongthe fast mode, the transition mode, and the slow mode based on themoving object information and determines the frame rate according to theselected mode. At this time, regarding the frame rate of the transitionmode, the FR control unit 104 performs adjustment so that the last frameof the transition mode becomes the I frame based on determinationinformation of I frame and P frame from the compression unit 102 and aset I frame cycle.

FIG. 6 is a flowchart showing a procedure of moving image compressionprocessing performed by the image processing apparatus 100 of thepresent embodiment. FIG. 6 is a flowchart showing an operation in thetransition mode that is different from the first embodiment. The imageprocessing apparatus 100 can implement each processing shown in FIG. 6when the CPU 11 reads and executes a necessary program.

First, in S21, the FR control unit 104 determines whether a frame to beprocessed is the first frame of the transition mode. When determiningthat the frame to be processed is the first frame, the FR control unit104 proceeds to S22 to perform initialization, and when determining thatthe frame to be processed is not the first frame, the FR control unit104 proceeds to S29.

In S22, the FR control unit 104 calculates a total number of frames tobe outputted in the transition mode based on the set I frame cycle.Specifically, the FR control unit 104 calculates the total number offrames N to be outputted in the transition mode by the following formulawhile assuming that an average of a total number of frames required froman I frame to the next I frame is m and a number of frames from an Iframe compressed in the past closest to a current-frame is n.N=m+n  (2)

Next, in S23, the FR control unit 104 determines whether the totalnumber of frames N calculated in S22 exceeds a preset threshold value A.When the total number of frames N is smaller than or equal to thethreshold value A, the FR control unit 104 proceeds to S24, and when thetotal number of frames N exceeds the threshold value A, the FR controlunit 104 proceeds to S25. Here, the threshold value A is, for example, avalue as shown below.A=ω _(h)−ω_(l)  (3)

Here, ω_(h) is the fast frame rate and ω_(l) is the slow frame rate.

In S24, the FR control unit 104 calculates the transition frame rate bylinear interpolation between the fast frame rate and the slow framerate. For example, the FR control unit 104 calculates a transition framerate ω_(k) at each time in the transition mode based on the followingformula.

$\begin{matrix}\left\lbrack {{Expression}\mspace{14mu} 2} \right\rbrack & \; \\{\omega_{k} = {{\frac{- \left( {\omega_{h} - \omega_{l}} \right)^{2}}{2N}t} + \omega_{l}}} & (4)\end{matrix}$

Here, t is time (sec) when start time of the transition mode is 0.

In S25, the FR control unit 104 determines whether the total number offrames N calculated in S22 exceeds a preset threshold value B. When thetotal number of frames N exceeds the threshold value B, the FR controlunit 104 proceeds to S26, and when the total number of frames N issmaller than or equal to the threshold value B, the FR control unit 104proceeds to S27. Here, the threshold value B is, for example, a value asshown below.

$\begin{matrix}\left\lbrack {{Expression}\mspace{14mu} 3} \right\rbrack & \; \\{B = \frac{\pi\left( {\omega_{h} - \omega_{l}} \right)}{2}} & (5)\end{matrix}$

In S26, the FR control unit 104 sets a timer for measuring time untiltime-out of the transition mode and proceeds to S27. Here, the timeuntil time-out is, for example, two seconds. When the total number offrames N exceeds the threshold value B, it is not possible to determinethe last frame using a set frame rate, so that a timer is set and aperiod of the transition mode is determined.

In S27, the FR control unit 104 calculates the transition frame rate byelliptic function interpolation between the fast frame rate and the slowframe rate. For example, the FR control unit 104 calculates thetransition frame rate ω_(k) at each time in the transition mode based onthe following formula.

$\begin{matrix}\left\lbrack {{Expression}\mspace{14mu} 4} \right\rbrack & \; \\{\omega_{k} = {\left( {\omega_{h} - \omega_{l}} \right)\sqrt{1 - {\frac{{\pi^{2}\left( {\omega_{h} - \omega_{l}} \right)}^{2}}{16N^{2}}t^{2}}}}} & (6)\end{matrix}$

The above formula (6) shows that the transition frame rate is temporallyvaried by connecting the fast frame rate and the slow frame rate with anelliptic function. When connecting by the elliptic function, the numberof frames in the transition mode is greater than that when connecting bya straight line, so that it is suitable when the number of frames to thenext I frame is large.

In S28, the FR control unit 104 obtains time from a current frame,determines time for compressing the next frame based on the transitionframe rate ω_(k) calculated in S24 or S27, and returns to S21.

In S29, the FR control unit 104 determines whether the frame to beprocessed is the last frame of the transition mode. When determiningthat the frame to be processed is not the last frame of the transitionmode, the FR control unit 104 proceeds to S30 and determines whether ornot the transition mode is time-out based on the timer set in S26. Whenthe transition mode is not time-out, the FR control unit 104 proceeds toS31, obtains time from the current frame, determines time forcompressing the next frame based on the transition frame rate ω_(k)calculated in S24 or S27, and returns to S21.

On the other hand, when determining that the frame to be processed isthe last frame of the transition mode in S29 or when determining thatthe transition mode is time-out in S30, the FR control unit 104 proceedsto S32 and performs end processing of the transition mode. Specifically,in S32, the compression control unit 105 creates an I frame and sets aquantization value, where the amount of code of the I frame is an amountof code to which the total code amount of the transition mode describedabove is appropriated, into the compression unit 102. A calculationmethod of the quantization value can be, for example, the same as thecalculation method of the quantization value for the last frame in thefirst embodiment.

In S33, the FR control unit 104 sets the frame rate to the slow framerate, transits from the transition mode to the slow mode, and ends theprocessing of FIG. 6.

As described above, in the same manner as in the first embodiment, theimage processing apparatus 100 according to the present embodimentcontrols the compression unit 102, suppresses the amounts of code offrames except for the last frame among the frames of the transitionmode, and appropriates the suppressed amounts of code for the lastframe. Here, the last frame is the frame (I frame) compressed by usingthe in-frame compression.

In this way, the last frame is the I frame whose image quality is betterthan that of the P frame that is compressed by using the interframeprediction, so that it is possible to more appropriately improve theimage quality at the slow frame rate.

Further, the image processing apparatus 100 has a unit to control thetransitional frame rate in the transition mode, so that the imageprocessing apparatus 100 can reliably cause the last frame of thetransition mode to be the I frame. As described above, in the presentembodiment, the frame immediately before the slow frame rate can be acompressed image using only the in-frame prediction without increasingthe entire amount of code of the video, so that it is possible toappropriately improve the image quality at the slow frame rate.

Third Embodiment

Next, a third embodiment of the present disclosure will be described.

In the third embodiment, when an actual compression efficiency is higherthan a preset target compression efficiency, an amount of codecorresponding to a difference between them is appropriated for the lastframe of the transition mode.

The configuration of the image processing apparatus according to thepresent embodiment is similar to that of the image processing apparatus100 shown in FIG. 1. However, processing of the compression control unit105 is different from that of the first embodiment. Therefore,processing different from that of the first embodiment will be mainlydescribed below.

FIG. 7 is a block diagram showing an example of a configuration of thecompression control unit 105 of the present embodiment. The compressioncontrol unit 105 includes a predictive code amount table updatefor-each-quantization-value unit 105 a, a predictive frame ratecomputing unit 105 b, a predictive code amount computing unit 105 c, acompression ratio computing unit 105 d, and a quantization valuedetermination unit 105 e.

Hereinafter, each block shown in FIG. 7 will be described with referenceto FIG. 8. FIG. 8 is a graph where the vertical axis represents anamount of code per second and the horizontal axis represents time. Asection A is an average holding period (predictive holding period) ofthe fast frame rate, and the section C is an average holding period(predictive holding period) of the slow frame rate. A section B is aperiod for improving the image quality of frame and is, for example, aperiod in which the number of frames is one. In FIG. 8, the fast framerate is represented by ω_(h) an average code amount of frames in thefast mode is represented by α_(h) the slow frame rate is represented byω_(l), an average code amount of frames in the slow mode is representedby α_(l), and a preset compression efficiency is represented by ε_(h).

The predictive code amount table update for-each-quantization-value unit(hereinafter referred to as a “table update unit”) 105 a holds a setquantization value and an amount of code of a frame compressed by thecompression unit 102 according to the quantization value as a table. Thetable update unit 105 a updates the table every time a frame iscompressed. An updated value may be a result of a moving average of 50times. Thereby, the amount of code is determined for each quantizationvalue. In FIG. 8, the average code amount α_(h) in the fast mode and theaverage code amount α_(l) in the slow mode are values provided by thetable update unit 105 a.

When the frame rate mode transits to the transition mode, the predictiveframe rate computing unit 105 b predicts a holding period of the slowmode to which the frame rate mode transits thereafter. Specifically, thepredictive frame rate computing unit 105 b predicts the holding periodof the slow mode from statistic information of the holding period of thepast slow mode determined by the FR control unit 104. For example, thepredictive frame rate computing unit 105 b may calculate an average ofthe past 50 holding periods of the slow mode by using a moving averageand use the average as a prediction value. The holding period of thepast slow mode is determined according to a detection frequency of amoving object in the video. Therefore, the predictive frame ratecomputing unit 105 b predicts the holding period of the slow mode basedon the statistic information of the moving object detection.

Further, the predictive frame rate computing unit 105 b can also predicta holding period of the fast mode in the same manner. In FIG. 8, valuesof the period A and the period C are values provided by predictive framerate computing unit 105 b.

The predictive code amount computing unit 105 c predicts the amount ofcode of the video distributed in the slow mode and outputs the predictedamount of code as a predictive code amount. Specifically, the predictivecode amount computing unit 105 c uses the average code amount α_(l) ofthe slow mode obtained from the table updated by the table update unit105 a, the slow frame rate ω_(l), and the holding period C of the slowmode predicted by the predictive frame rate computing unit 105 b. InFIG. 8, a value of an area D shown by oblique lines is a value providedby the predictive code amount computing unit 105 c.

The compression ratio computing unit 105 d predicts an actualcompression efficiency when the amount of code of the video iscompressed to the predictive code amount in the predicted holding periodof the slow mode. Specifically, the compression ratio computing unit 105d calculates a ratio between a first code amount when the fast mode isused at all times in the periods A to C in FIG. 8 and a second codeamount when the fast mode is used in the periods A and B and the slowmode is used in the period C as the actual compression efficiency.

The compression ratio computing unit 105 d can calculate the first codeamount by using the average code amount α_(h) in the fast mode obtainedfrom the table described above, the fast frame rate ω_(h), the averageholding period A of the fast mode, and the average holding period C ofthe slow mode. Further, the compression ratio computing unit 105 d cancalculate the second code amount by using the average code amount α_(h)in the fast mode obtained from the table described above, the fast framerate ω_(h), the average holding period A of the fast mode, and thepredictive code amount computed by the predictive code amount computingunit 105 c.

The quantization value determination unit 105 e compares the actualcompression efficiency calculated by the compression ratio computingunit 105 d with a predetermined target compression efficiency anddetermines the quantization value for the last frame of the transitionmode based on a result of the comparison. First, when the actualcompression efficiency is higher than the target compression efficiency,the quantization value determination unit 105 e computes an amount ofcode to be assigned to the last frame of the transition frame accordingto a difference between them and calculates an added amount of codewhere the amount of code to be assigned to the last frame of thetransition frames is added to the average code amount α_(h) in the fastmode. Next, the quantization value determination unit 105 e calculates aquantization value where an amount of code becomes the added amount ofcode as a result of compression of the last frame from the table, anddetermines the calculated quantization value as the quantization valuefor the last frame of the transition mode.

Specifically, the quantization value determination unit 105 e calculatesa code amount X that satisfies the following formula and calculates aquantization value that satisfies the calculated code amount X as thequantization value for the last frame.ε_(h)α_(h)ω_(h)(A+B+C)=Aα _(h)ω_(h) +BX+Cα _(l)ω_(l)  (7)

In the present embodiment, the quantization value determination unit 105e determines a difference between the amount of code of video when thecompression efficiency becomes the target compression efficiency in theperiod (A+B+C) and the second code amount as the amount of code to beassigned to the last frame.

FIG. 9 is a flowchart showing a procedure of moving image compressionprocessing performed by the image processing apparatus 100 of thepresent embodiment. The moving image compression processing isprocessing that improves image quality in the slow mode when performingframe rate control that changes the frame rate according to the presenceor absence of a moving object in a video. The processing of FIG. 9 isstarted at a timing when the image processing apparatus 100 is started.However, a start timing of the processing of FIG. 9 is not limited tothe aforementioned timing. The image processing apparatus 100 canimplement each processing shown in FIG. 9 when the CPU 11 reads andexecutes a necessary program.

First, in S41, the FR control unit 104 performs initialization andselects the fast mode as the frame rate mode.

Next, in S42, the FR control unit 104 is inputted with the moving objectinformation from the moving object detection unit 103 and determineswhether a moving object is detected from a video. When the moving objectis detected, the FR control unit 104 proceeds to S43, and when no movingobject is detected, the FR control unit 104 proceeds to S48. In S43, thecompression control unit 105 updates a moving object detection frequencyand proceeds to S44.

In S44, the compression control unit 105 predicts a holding period ofthe slow mode based on the moving object detection frequency. At thistime, the compression control unit 105 also predicts a holding period ofthe fast mode. In S45, the compression control unit 105 updates a tablewhere the average code amount and the quantization value are associatedwith each other, and proceeds to S46. In S46, the compression controlunit 105 sets the quantization value for the fast frame rate to thecompression unit 102 and proceeds to S47. In S47, the FR control unit104 sets the frame rate to the fast frame rate, transits to the fastmode, and returns to S42.

In S48, the FR control unit 104 determines whether the current framerate mode is the fast mode. When determining that the current frame ratemode is the fast mode, the FR control unit 104 proceeds to S49, and whendetermining that the current frame rate mode is not the fast mode, theFR control unit 104 proceeds to S51. In S49, the FR control unit 104transits from the fast mode to the transition mode. At this time, the FRcontrol unit 104 initializes the transition mode and sets the frame rateto the transition frame rate. In S50, the FR control unit 104 starts atimer for measuring time until time-out of the transition mode, andreturns to S42.

In S51, the compression control unit 105 updates a table of an averagecode amount of compressed frames and the quantization value. In S52, theFR control unit 104 determines whether the current frame rate mode isthe transition mode. When determining that the current frame rate modeis the transition mode, the FR control unit 104 proceeds to S53, andwhen determining that the current frame rate mode is not the transitionmode, the FR control unit 104 returns to S42.

In S53, the FR control unit 104 determines whether the transition modeis time-out. When the transition mode is not time-out, the FR controlunit 104 proceeds to S54.

In S54, the compression control unit 105 calculates a predictive codeamount in the slow mode based on information obtained from the updatedtable and the holding period of the slow mode predicted in S44, andproceeds to S55.

In S55, the compression control unit 105 calculates an actualcompression efficiency by using the predictive code amount, and proceedsto S56.

In S56, the compression control unit 105 compares the actual compressionefficiency calculated in S55 with a predetermined target compressionefficiency. When the compression control unit 105 determines that theactual compression efficiency is higher than the target compressionefficiency, the compression control unit 105 assigns an amount of codecorresponding to a difference between them to the last frame of thetransition mode and calculates an amount of code for the last frame.Further, the compression control unit 105 calculates a quantizationvalue from the calculated amount of code for the last frame, updates thequantization value as the quantization value for the last frame of thetransition mode, and returns to S42.

On the other hand, when determining that the transition mode is time-outin S53, the FR control unit 104 proceeds to S57. In S57, the compressioncontrol unit 105 sets the quantization value for the last frame to thecompression unit 102 and returns to S58. In S58, after completingcompression of the last frame, the compression control unit 105 updatesthe table of the average code amount and the quantization value andproceeds to S59.

In S59, the compression control unit 105 sets the quantization value forthe slow frame rate to the compression unit 102 and proceeds to S60. InS60, the FR control unit 104 sets the frame rate to the slow frame rate,transits to the slow mode, and returns to S42.

As described above, the image processing apparatus 100 according to thepresent embodiment predicts the holding period of the fast mode and theholding period of the slow mode based on the statistic information ofthe moving object detection. Further, the image processing apparatus 100predicts the compression efficiency of the video based on the predictedholding periods of the fast mode and the slow mode. When the predictedcompression efficiency is higher than the target compression efficiency,the image processing apparatus 100 determines an amount of code to beappropriated for the last frame of the transition mode based on adifference between the compression efficiency and the target compressionefficiency.

In this way, the compression efficiency is allowed to be lowered to thetarget compression efficiency, and the amount of code of the last frameof the transition mode is increased. Specifically, the amount of code ofthe last frame of the transition mode can be increased to be greaterthan the specified value within a range where the entire amount of codeof the video does not exceed the amount of code (the target amount ofcode) when the compression efficiency becomes the target compressionefficiency. Therefore, it is possible to improve the image quality atthe slow frame rate while maintaining the reduction of the entire amountof code of the video by the frame rate control.

The image processing apparatus 100 computes a ratio between the firstcode amount in a case when the video is distributed at the fast framerate in the period (A+B+C) of FIG. 8 and the second code amount in acase when the video is distributed at the fast frame rate in the period(A+B) and the video is distributed at the slow frame rate in the periodC. The image processing apparatus 100 predicts the computed value as theactual compression efficiency.

In this way, the image processing apparatus 100 predicts the holdingperiod of the fast mode (the period A) and the holding period of theslow mode (the period C) based on the statistic information of themoving object detection, so that the image processing apparatus 100 canappropriately predict the holding period of each mode and can accuratelypredict the compression efficiency. Therefore, it is possible toappropriately determine the amount of code to be appropriated for thelast frame of the transition mode.

Here, the image processing apparatus 100 can determine a differencebetween the amount of code of the video in a case when the compressionefficiency becomes the target compression efficiency in the period(A+B+C) and the second code amount as the amount of code to beappropriated for the last frame. In this case, it is possible toappropriate the amount of code for the last frame of the transition modeso that the entire amount of code of the video becomes the target amountof code. Therefore, it is possible to apply the quantization value ofthe highest image quality within a range of allowable compressionefficiency to the last frame of the transition mode, so that it ispossible to more appropriately improve the image quality at the slowframe rate.

Fourth Embodiment

Next, a fourth embodiment of the present disclosure will be described.

In the fourth embodiment, an image is divided into a plurality of areasand the frame rate is controlled for each divided partial area.

In the present embodiment, when the detection frequency of a movingobject varies for each partial area, an effective frame rate of eacharea is controlled by reducing or eliminating an image code amount foreach partial area, so that it is possible to reduce the amount of code.Further, in the present embodiment, in the same manner as in the thirdembodiment described above, the amount of code is appropriated for thelast frame of the transition mode according to a difference between theactual compression efficiency and the target compression efficiency inorder to improve the image quality of partial areas in the slow mode.

It is possible to reduce or eliminate the image code amount in eachpartial area by, for example, forcibly inserting a skip macroblock ofH.264. An area segmentation in a screen may be predetermined. Forexample, as shown in FIG. 10, an image 400 may be divided into fourareas.

In the image 400, an area A and an area B are areas to be backgroundimages, and an appearance ratio of a moving object is low in theseareas. An area C is an area including an intersection. The appearanceratio of a moving object is high in the area C. The appearance ratio ofa moving object in the area D is lower than that in the area C. In thiscase, in the area A and the area B, the slow mode is set and, forexample, the frame rate is effectively set to 0.01 fps. In the area Cand the area D, the fast mode is set. For example, the frame rate is setto 30 fps in the area C, and the frame rate is set to 15 fps in the areaD.

The configuration of the image processing apparatus according to thepresent embodiment is the same as that of the image processing apparatusaccording to the third embodiment and is the same as that of the imageprocessing apparatus 100 shown in FIG. 1. However, processing of thecompression control unit 105 is different from that of the thirdembodiment. Therefore, processing different from that of the thirdembodiment will be mainly described below.

FIG. 11 is a block diagram showing an example of a configuration of thecompression control unit 105 according to the present embodiment. Thecompression control unit 105 includes an area-based predictive codeamount table update for-each-quantization-value unit 105 a′, anarea-based predictive frame rate computing unit 105 b′, a predictivecode amount computing unit 105 c′, and a compression ratio computingunit 105 d′. The compression control unit 105 further includes anarea-based image quality weight setting unit 105 f and a quantizationvalue determination unit 105 e′.

Processing of the area-based predictive code amount table updatefor-each-quantization-value unit 105 a′, the area-based predictive framerate computing unit 105 b′, the predictive code amount computing unit105 c′, and the compression ratio computing unit 105 d′ corresponds tothat of each block 105 a to 105 d shown in FIG. 7 except for differencesof areas. Processing of the quantization value determination unit 105 e′corresponds to that of the quantization value determination unit 105 eshown in FIG. 7 except for differences of areas.

The area-based image quality weight setting unit 105 f compares theactual compression efficiency calculated by the compression ratiocomputing unit 105 d′ with a predetermined target compressionefficiency. When the actual compression efficiency is higher than thetarget compression efficiency, the area-based image quality weightsetting unit 105 f assigns an amount of code corresponding to adifference between them to the last frame of the transition mode andcalculates an amount of code where the amount of code to be assigned tothe last frame of the transition frames is added to the amount of codein the fast mode.

In this case, when there are areas that become the transition mode atthe same time, amounts of code may be equally assigned to these areas orthe amounts of code assigned to these areas may be weighted. Forexample, dispersions of code amounts for the past 30 frames of each areamay be calculated, and weight may be set so that an amount of codeinversely proportional to the dispersion of the amount of code isassigned. In other words, the weight may be set based on a ratio of theinverse of the dispersion of the amount of code.

As described above, the image processing apparatus 100 according to thepresent embodiment determines the frame rate for each partial areaobtained by dividing a flame into a plurality of areas. When the imageprocessing apparatus 100 distributes a video at a frame rate determinedfor each partial area, the image processing apparatus 100 simulativelycontrols the frame rate for each partial area by inserting a skipmacroblock into the partial area. Thereby, it is possible to effectivelycontrol the frame rate of each area.

Further, the image processing apparatus 100 controls the compressionunit 102 and determines the amount of code of the last frame among theframes of a partial area distributed at the transitional frame ratebased on a ratio of the inverse of the dispersion of the amount of codeof each partial area. Thereby, it is possible to set a quantizationvalue for achieving higher image quality on an area having the lowestdispersion and converge a code amount dispersion difference of eacharea.

Other Embodiments

The present invention can be realized by processing where a programrealizing one or more functions of the embodiments described above issupplied to a system or an apparatus through a network or a storagemedium and one or more processors in a computer of the system or theapparatus reads and executes the program. Alternatively, the presentinvention can be realized by a circuit (for example, ASIC) that realizesone or more functions.

According to the embodiments described above, it is possible to suppressdegradation of video image quality when the frame rate is controlled toa second frame rate lower than a first frame rate while the reduction ofthe entire amount of code of the video is realized by the frame ratecontrol.

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2018-165065, filed Sep. 4, 2018, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A video data generation apparatus that performsgeneration processing for generating video data, the video datageneration apparatus comprising: a frame rate determination unitconfigured to determine a frame rate of the video data; a code amountdetermination unit configured to determine a target code amount that canbe used in each frame of the video data; and an encoding unit configuredto perform the generation processing for generating the video dataaccording to the frame rate determined by the frame rate determinationunit and the code amount determined by the code amount determinationunit, wherein when the frame rate determination unit changes, in a firstframe, the frame rate of the video data from a first frame rate to asecond frame rate lower than the first frame rate, the code amountdetermination unit makes at least a target code amount in a frameimmediately before the first frame smaller than when the frame rate isnot changed from the first frame rate to the second frame rate in thefirst frame, and makes a target code amount in the first frame greaterthan when the frame rate is not changed from the first frame rate to thesecond frame rate in the first frame.
 2. The video data generationapparatus according to claim 1, further comprising: a detection unitconfigured to perform moving object detection processing that detects amoving object from a video, wherein the frame rate determination unitdetermines the frame rate based on a result of the moving objectdetection processing performed by the detection unit.
 3. The video datageneration apparatus according to claim 2, wherein the moving objectdetection processing is processing that uses a background differencemethod and the detection unit determines that a foreground detected bythe moving object detection processing is a moving object.
 4. The videodata generation apparatus according to claim 2, wherein when changingthe frame rate in the video data from a predetermined frame rate higherthan the first frame rate to the second frame rate according to theresult of the moving object detection processing, the frame ratedetermination unit changes the frame rate from the predetermined framerate to the second frame rate, and thereafter changes the frame ratefrom the second frame rate to the first frame rate.
 5. The video datageneration apparatus according to claim 2, wherein the frame ratedetermination unit determines a first period in which the video data isgenerated at the first frame rate and a second period in which the videodata is generated at the second frame rate based on statisticinformation pertaining to the result of the moving object detectionprocessing.
 6. The video data generation apparatus according to claim 1,wherein the frame rate determination unit determines the frame rate foreach partial area obtained by dividing the flame into a plurality ofareas.
 7. The video data generation apparatus according to claim 6,wherein the encoding unit changes a frame rate for each partial area byinserting a skip macroblock into the partial area according to the framerate determined by the frame rate determination unit.
 8. The video datageneration apparatus according to claim 1, wherein the first frame isencoded by using an in-frame prediction.
 9. The video data generationapparatus according to claim 1, further comprising: a distribution unitconfigured to distribute the video data.
 10. A video data generationmethod that performs generation processing for generating video data,the video data generation method comprising: a frame rate determinationstep of determining a frame rate of the video data; a code amountdetermination step of determining a target code amount that can be usedin each frame of the video data; and an encoding step of performing thegeneration processing for generating the video data according to theframe rate determined in the frame rate determination step and the codeamount determined in the code amount determination step, wherein whenthe frame rate of the video data is changed from a first frame rate to asecond frame rate lower than the first frame rate in a first frame inthe frame rate determination step, in the code amount determinationstep, at least a target code amount in a frame immediately before thefirst frame is made smaller than when the frame rate is not changed fromthe first frame rate to the second frame rate in the first frame, and atarget code amount in the first frame is made greater than when theframe rate is not changed from the first frame rate to the second framerate in the first frame.
 11. A non-transitory recording medium thatrecords a program for causing a computer to perform a video datageneration method of performing generation processing for generatingvideo data, the method comprising: a frame rate determination step ofdetermining a frame rate of the video data; a code amount determinationstep of determining a target code amount that can be used in each frameof the video data; and an encoding step of performing the generationprocessing for generating the video data according to the frame ratedetermined in the frame rate determination step and the code amountdetermined in the code amount determination step, wherein when the framerate of the video data is changed from a first frame rate to a secondframe rate lower than the first frame rate in a first frame in the framerate determination step, in the code amount determination step, at leasta target code amount in a frame immediately before the first frame ismade smaller than when the frame rate is not changed from the firstframe rate to the second frame rate in the first frame, and a targetcode amount in the first frame is made greater than when the frame rateis not changed from the first frame rate to the second frame rate in thefirst frame.